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Abstract 

We consider the monotone submodular fc-set packing problem in the context of the more 
general problem of maximizing a monotone submodular function in a fc-exchange system. These 
systems, introduced by Feldman et al. [5], generalize the matroid k-parity problem in a wide 
class of matroids and capture many other combinatorial optimization problems. We give a 
deterministic, non-oblivious local search algorithm that attains an approximation ratio of (k + 
3)/2 + e for the problem of maximizing a monotone submodular function in a fc-exchange system, 
improving on the best known result of k + e, and answering an open question posed in Feldman 
et al. 

1 Introduction 

In the general k-set packing problem, we are given a collection Q of sets, each with at most k 
elements, and an objective function / : lP — > M + assigning each subset of Q a value, and seek a 
collection of pairwise-disjoint sets S C Q that maximizes /. In the special case that f(A) = \A\, 
we obtain the unweighted k-set packing problem. Similarly, if / is linear function, so that f(A) = 
^2eeA w ( e ) f° r some weight function w : Q — > M + we obtain the weighted k-set packing problem. In 
this paper we consider the case in which / may be any monotone submodular function. 

For unweighted k-set packing, Hurkens and Schrijver |15j and Halldorsson [33] independently 
obtained a k/2 + e approximation via a simple local search algorithm. Using similar techniques, 
Arkin and Hassin [1] obtained a k — 1 + e approximation for weighted k-set packing, and showed that 
this result is tight for their simple local search algorithm. Chandra and Halldorsson [5] showed that 
a more sophisticated local search algorithm, which starts with a greedy solution and always chooses 
the best possible local improvement at each stage, attains an approximation ratio of 2(k + l)/3 + e. 
This was improved further by Berman [2], who gave a non-oblivious local search algorithm yielding 
a (k + l)/2 + e approximation for weighted k-set packing. Non-oblivious local search [16] is a 
variant of local search in which an auxiliary objective function to evaluate solutions, rather than 
the problem's given objective. In the case of Berman, the local search procedure repeatedly seeks 
to improve the sum of square of the weights in the current solution, rather than the sum of the 
weights. 

Many of the above local search algorithms for k-set packing yield the same approximations for 
the more general problem of finding maximum independent sets in (k + l)-claw free graphs. Addi- 
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tionally, local search techniques have proved promising for other generalizations of k-set packing, 
including variants of the matroid fc-parity problem [181 120j . Motivated by the similarities between 
these problems, Feldman et al. [9] introduced the class of k-exchange systems, which captures prob- 
lems amenable to approximation by local search algorithms. These systems are formulated in the 
general language of independence systems, which we now briefly review. 

An independence system is specified by a ground set G, and a hereditary (i.e. non-empty and 
downward-closed) family I of subsets of Q. These subsets of Q contained in I are called independent 
sets, and the inclusion- wise maximal sets of I are called bases of the independence system (I, Q). 
Given an independence system (G,T) and a function / : lP — > R + , we consider the problem of 
finding an independent set S € I that maximizes /. 

The class of A:-exchange systems satisfy the following additional property: 

Definition 1.1 (/c-exchange system |9j). A hereditary system I is a k-exchange system if, for all 
A and B in X, there exists a multiset Y = {Y e C B\A \ e 6 A\ B}, containing a subset Y e of B \ A 
for each element e 6 A \ B, that satisfies: 

(Kl) \Y e \ < k for each x G A. 

(K2) Every x € B \ A appears in at most k sets of Y. 
(K3) For all C C A \ B, {B \ (\J eeC Y e ) ) U C £ 1. 

We call the set Y e in Definition 1 1 . 1 1 the neighborhood of e in B. For convenience, we extend the 
collection Y in Definition 11.11 by including the set Y x = {x} for each element iGinB. It is easy 



to verify that the resulting collection still satisfies conditions (K1) -(K3) 

The 1-exchange systems are precisely the class of strongly base orderable matroids described 
by Brualdi [3]. This class is quite large and includes all gammoids, and hence all transversal and 
partition matroids. For k > 1, the class of ^-exchange systems may be viewed as a common gener- 
alization of the matroid /c-parity problem in strongly base orderable matroids and the independent 
set problem in (k + l)-claw free graphs. Feldman et al. showed that £>exchange systems encompass 
a wide variety of combinatorial optimization problems, including as k-set packing, intersection of 
k strongly base orderable matroids, hypergraph 6-matching (here k = 2), as well as problems such 
as asymmetric traveling salesperson (here k = 3). 

Our results hold for any /c-exchange system, and so we present them in the general language 
of Definition 11.11 However, the reader may find it helpful to think in terms of a concrete problem, 
such as the k-set packing problem. In that case, the ground set Q is the given collection of sets, and 
a sub-collection of sets S C Q is independent if and only if all of the sets in S are disjoint. Given 
A and B as in Definition 11.11 Y e is the set of all sets in B that contain any element contained by 
the set e £ A (i.e. the set of all sets in B that are not disjoint from e). Then, property |(K3)| is 
immediate, and |(K1)| and |(K2)| follow directly from the fact that each set in Q contains at most k 
elements. 



1.1 Related Work 

Recently, the problem of maximizing submodular functions subject to various constraints has at- 
tracted much attention. We focus here primarily on results pertaining to matroid constraints and 
related independence systems. 

In the case of an arbitrary single matroid constraint, Calinescu et al. have attained a e/(e — 1) 
approximation for monotone submodular maximization, via the continuous greedy algorithm. This 
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result is tight, provided that P ^ NP [6]. In the case of k > 2 simultaneous matroid constraints, 
an early result of Fisher, Nemhauser, and Wolsey [10J shows that the standard greedy algorithm 
attains a k + 1 approximation for monotone submodular maximization. Fischer et al. state further 
that the result can be generalized to fc-systems (a full proof appears in Calinescu et al. [4]). More 
recently, Lee, Sviridenko, and Vondrak [19] have improved this result to give a k + e approximation 
for monotone submodular maximization over k > 2 arbitrary matroid constraints, via a simple, 
oblivious local search algorithm. A similar analysis was used by Feldman et al. [9] to show that 
oblivious local search attains a k + e approximation for the class of A;-exchange systems (here, 
again, k > 2). For the more general class of fe-systems, Gupta et al. |12| give a (1 + f3)(k + 2 + 1/&) 
approximation, where (3 is the best known approximation ratio for unconstrained non-monotone 
submodular maximization. 

In the case of unconstrained non-monotone submodular maximization, Feige, Mirrokni, and 
Vondrak [7] gave a randomized 2.5 approximation, which was iteratively improved by Gharan and 
Vondrak [11] and then Feldman, Naor, and Shwartz [8] to ~ 2.38. For non-monotone maximization 
subject to k matroid constraints, Lee, Sviridenko, and Vondrak [17] gave a k + 2 + 1/k + e approx- 
imation, and later improved [19] this to a k + 1 + l/(k — 1) + e approximation. Again, the latter 
result is obtained by a standard local search algorithm. Feldman et al. [9] apply similar techniques 
to yield a k + 1 + l/(k — 1) + e approximation for non-monotone submodular maximization the 
general class of /c-exchange systems. 

1.2 Our Contribution 

In the restricted case of a linear objective function, Feldman et al. [9] gave a non-oblivious local 
search algorithm inspired by Berman's algorithm [2] for (k + l)-claw free graphs. They showed 
that the resulting algorithm is a (k + l)/2 + e approximation for linear maximization in any k- 
exchange system. Here we consider a question posed in [9]: namely, whether a similar technique 
can be applied to the case of monotone submodular maximization in ^-exchange systems. In this 
paper, we give a successful application of the non-oblivious local search techniques to the case of 
monotone submodular maximization in a A;-exchange system. As in [9], the /c-exchange property 
is used only in the analysis of our algorithm. The resulting non-oblivious local search algorithm 
attains an approximation factor of + e. For k > 3, this improves upon the k + e approximation 
obtained by the oblivious local search algorithm presented in [9j. Additionally, we note that our 
algorithm runs in time polynomial in e , while the k + e approximation algorithm of [9] requires 
time exponential in e . 

As a consequence of our general result, we obtain an improved approximation guarantee of 
for a variety of monotone submodular maximization problems (some of which are generalizations of 
one another) including: fc-set packing, independent sets in (k + l)-claw free graphs, A:-dimensional 
matching, intersection of k strongly base orderable matroids, and matroid fc-parity in a strongly 
base orderable matroid. In all cases, the previously best known result was k + e. 

2 A First Attempt at the Submodular Case 

Before presenting our algorithm, we describe some of the difficulties that arise when attempting to 
adapt the non-oblivious local search algorithm of [9] and [2] to the submodular case. Our hope is 
that this will provide some intuition for our algorithm, which we present in the next section. 
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We recall that a function / : 2 G -> M + is submodular if /(A) + f(B) > f{A U B) + f(A n 5) 
for all A,B QQ. Equivalently, / is submodular if for all S C T and all x & T, f(S + x) - f(S) > 
f(T + x) — f(T). In other words, submodular functions are characterized by decreasing marginal 
gains. We say that a submodular function / is monotone if it additionally satisfies f(S) < f(T) 
for all SCT. 

The non-oblivious algorithm of [9] for the linear case is shown in Algorithm [TJ It proceeds 
from one solution to another by applying a k -replacement (A,B). Formally, we call the pair of 
sets (A, B), where A C Q \ {S \ B) and BCSa fc-replacement if \A\ < k, \B\ < k 2 — k + 1 and 
(S \ B) U A € X. Algorithm Q] repeatedly searches for a ^-replacement that improves an auxiliary 
potential function w 2 . Because / is linear, it can be represented as a sum of weights, one for each 
element in S. If w(e) = /({e}) is the weight assigned to an element e in this representation, then 
the non-oblivious potential function is given by w 2 (S) = Y^ees w ( e ) 2 • That is, our non oblivious 
potential function w 2 (S) is simply the sum of the squared weights of the elements of 



Algorithm 1: Non-Oblivious Local Search for Linear Objective Functions 
Input: • Ground set Q 

• Membership oracle for X C 2? 

• Value oracle for monotone submodular function / : 2^ — > M.+ 

• Approximation parameter e € (0, 1) 

Let Sinn = {argmax eg g w(e)}; 
Let a = w(Si n u)e/n; 

Round all weights w{e) down to integer multiples of a; 

Sold 4 S\ 

repeat 

foreach k -replacement (A, B) do 
if w 2 (A) > w 2 {B) then 

Sold ^~ S; 

S^{S\B)UA; 

break; 

until S i d = S; 
return S; 



In the monotone submodular case, we can no longer necessarily represent / as a sum of weights. 
However, borrowing some intuition from the greedy algorithm, we might decide to replace each 
weight w(e) in the potential function w with the marginal gain/loss associated with e. That is, 
at the start of each iteration of the local search algorithm, we assign each element e G Q weight 
w(e) = f(S + e) — f(S — e), where S is the algorithm's current solution, then we proceed as before. 
Note that w(e) is simply the marginal gain attained by adding e to S, in the case that e ^ S or the 
marginal loss suffered by removing e from S, in the case that e £ S. We define the non-oblivious 
potential function w 2 in terms of the (current) weight function w, as before. 

1 To ensure polynomial-time convergence, Algorithm [T] first round the weights down to integer multiples of a 
suitable small value a, related to the approximation parameter e. The algorithm then converges in time polynomial 
in e _1 and n, at a loss of only (1 — e) _1 in the approximation factor. 
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Unfortunately, the resulting algorithm may fail to terminate, as the following small example 
shows. We consider a simple, unweighted coverage function on the universe U = {a, 6, c, x, y, z}, 
defined as follows. Let: 



We define Q = {1, 2, 3,4} and f(A) = \\J ieA SA for all ACQ. Finally, we consider the 2-exchange 
system with only 2 bases: P = {1,2} and Q = {3,4}. For current solution S = P we have 
w{\) = w(2) = 1 and w(3) = w(4) = 2. Since w 2 ({l,2}) = 2 < 8 = u> 2 ({3,4}), the 2-replacement 
({3, 4}, {1, 2}) is applied, and the current solution becomes Q. In the next iteration, we have S = Q, 
and w(l) = w{2) = 2 and w(3) = u?(4) = 1, so the 2-replacement ({1, 2}, {3, 4}) is applied by the 
algorithm. This returns us to the solution to P, where the process repeats indefinitely. 

3 The New Algorithm 

Intuitively, the problem with this initial approach stems from the fact that the weight function used 
at each step of the algorithm depends on the current solution S (since all marginals are taken with 
respect to S). Each time the algorithm makes an improvement, it changes the current solution, 
thereby changing the weights assigned to all elements in the next iteration. Hence, we are effectively 
making use of an entire family of non-oblivious potential functions, indexed by the current solution 
S. It may be the case that a /c-replacement (^4, B) results in an improvement with respect to the 
current solution's potential function, but in fact results in a decreased potential value in the next 
iteration, after the weights have been updated. 

Surprisingly, we can solve the problem by introducing even more variation in the potential 
function. Specifically, we allow the algorithm to use a different weight function not only for each 
current solution S, but also for each k-replacement (A, B) that is considered. We give the full 
algorithm at the end of this section and a detailed analysis in the next. 

Our general approach is to consider the elements of a set X in some order -<, and assign to 
each e G X in the set the marginal gain in / obtained when it is added to the set containing all 
the preceding elements of X. By carefully updating -< together with the current solution S at each 
step, we ensure that the algorithm converges to a local optimum. We now give the details for how 
are weights are calculated, given the current ordering -<. 

At each iteration of the algorithm, before searching an improving fc-replacement, we assign 
weights w to all of the elements of the current solution S. The weights will necessarily depend on 
S, but remain fixed for all /c-replacements considered in the current phase. Let Sj be the ith element 
of S in the ordering -< and let Si = {sj G S : j < i} be the set containing the first i elements of S 
in the ordering -<. Then, the weight function w : S — >■ R + is given by 



5 1 = {a,b} 

5 2 = {a, c} 



53 = {x,y} 

54 = {x,z} 



w{Si) = /{Si-! + Si) - f{Si-l) 



for all Si G S. Note that our weight function satisfies 



\s\ 




(1) 



5 



In order to evaluate each /c-replacement (A,B), we need to assign weights to the elements in 
A C Q \{S \B). We use a different weight function for each /c-replacement. Suppose that we are 
considering the /c-replacement (A,B). Let be the ith element of A in the ordering -< and let 
Ai = {a,j E A : j < i} be the set containing the first % elements of A in the ordering -<. Then, we 
define the weight function W(a,b) '■ A — > M+ by 

W (A,B) (°t) = f((S \ B) U A^x + ai ) - f((S \ B) U 

for all a% £ A. Note that for every /c-replacement (^4, B), 

\A\ 

p E > E (/(( 5 \ B ) u ^-i + °o - \ 5 ) u ^-0) 

zsA i=l 

= f((S\B)uA)-f(S\B)>f(SuA)-f(S) , (2) 

where the last inequality follows from the decreasing marginals characterization of submodularity. 
Note that since the function / is monotone submodular, all of the weights w and Wi^b) that we 
consider will be nonnegative. This fact plays a crucial role in our analysis. 

Our final algorithm appears in Algorithm[2j We start from an initial solution Si n it = {argmax ee g /({e})}, 
consisting of the singleton set of largest value. Note that after applying a /c-replacement (A, B), the 
algorithm updates -< to ensure that all of the elements of S\B precede those of A. As we shall see in 
the next section, this ensures that the algorithm will converge to a local optimum. As in the linear 
case, we use the non-oblivious potentials w 2 {B) = ^2 beB w(b) 2 and w^b)(^) = J2 a eA W (A,B) ( a ) 2 - 
Again, we note that while all of our weights implicitly depend on the current solution, the weights 
W (A,B) additionally depend on the /c-replacement (A, B) considered. 

Additionally, to ensure polynomial-time convergence, we round all of our weights down to 
the nearest integer multiple of a, depending on the parameter e. This will ensure that every 
improvement improves the current solution by an additive factor of at least a 2 . Because of this 
rounding factor, we must actually work with the following analogs of ([I]) and ([2]): 

|S| |S| 

E < E (/os«-i + s *) - /osw)) = E (/(so - /(*-0) = /( s ) - /( ) ^ /( 5 ) ( 3 ) 

xG5 i=l i=l 

E w (a,b){x) > E (f(( s \ B ) u A i-i + a i) - f(( s \ B ) U A i~l) - «) 

x£A i=l 

= f((S \B)UA)- f(S \ B) - \A\a > f(S U A) - f(S) - \A\a (4) 
4 Analysis of Algorithm [2] 

We now analyze the approximation and runtime performance of Algorithm [2j We now consider the 
worst-case ratio, or locality gap, f(0)/f(S) where S is any locally optimal solution (with respect to 
Algorithm [2js potential function) and O is a globally optimal solution. We shall need the following 
technical lemma, which is a direct consequence of Lemma 1.1 in [19]. We give a proof here for the 
sake of completeness. 
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Algorithm 2: Non-Oblivious Local Search 
Input: • Ground set Q 

• Membership oracle for X C lP 

• Value oracle for monotone submodular function / : iP — > R + 

• Approximation parameter e € (0, 1) 

Let S^it = {argmax ee g/({e})} 

Let 5 = (1 + and a = f(S init )8/n 

S <— Si n it, ~< <— an arbitrary total ordering on Q, and S id <— S 
repeat 

Sort S according to -< and let Sj be the ith element in S 
X <- 

for i = 1 to |5| do 

<- L(/(* + «i) - /P0)A*J a 
_ X ^X + Si 

foreach k replacement (A, B) do 

Sort A according to -< and let be the ith element in A 

X ^S\B 

for i = 1 to \A\ do 

W (A,B) ifli) 4r- l(f(X + ai ) - f(X))/a\ a 

_ X <r- X + Oi 

if wh B JA) > w 2 (S) then 



until S i d = S 
return S 



Lemma 4.1. Let f be a submodular function on Q, Let T,S C Q, and {Tj}* =1 be a partition of 
T\S. Then, 




fox all x e S\B, y £ A 
if x ~< y for all other x, y 



break 



£ (/(S U Tj) - /(£?)) > /(5 U T) - 



i=i 



Proof. Define Aq = S and ^ 



4 = Ti U A-i for all 1 < i < t. Suppose that T { = {tj}^. Then, note 
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that S C Ai-i and Tj n Ai—i = 0. Submodularity of / implies that: 

Ti 



m us)- f(s) = e {fmiS us) - /({tiiti u s) 



T 

> E (/(Mi 1 u - /(^H =1 u ^ 

= /(^ U A^ x ) - f(Ai-i) = f(Ai) - 

Now, we have 

t t 

E U T *) - H S » > E (/(^) " /(^-i)) = /(^) - /(^o) = /(S U T) - f(S) □ 
1=1 1=1 

We begin by considering the approximation ratio of Algorithm [2J Suppose that 5 is the locally 
optimal solution returned by the algorithm on some instance, while O is a global optimum for 
this instance. Then, for every fc-replacement (A,B), we must have w 2 AB ^{A) < w 2 (B), where w 
and each wr^m are weight functions determined by the solution S. We consider only a particular 
subset of /c-replacements in our analysis. 

We have S, O € X for the ^-exchange system X. Thus, there must be a collection Y assigning 
each e of O a neighborhood Y e C. S, satisfying the conditions of Definition 11.11 For each x € S, let 
P x be the set of all elements ine G O for which: (1) x € Y e and (2) for all z £ 1^, w(z) < w(x). That 
is, P x is the set of all elements of O in which x is the heaviest element. Note that the construction 
of P x depends on the fact that the weights w assigned to elements in S are fixed throughout each 
iteration, and do not depend on the particular improvement under consideration. 

We define N x = \J eeP , r Y e , and consider (P X ,N X ). Property [(K2)] of Y ensures that \P X \ < k. 



Similarly, property (Kl) , together with the fact that all elements e 6 P x have as a common 
neighbor x £ Y e , ensures that |iVa.| < 1 + k(k — 1) = k 2 — k + 1. Finally, property | (K3) | ensures that 
(S \ N x ) U P x € X. Thus, (P X ,N X ) is a valid fc-replacement for all sets P x C O, x € S. Observe 
that {P x }xeS is a partition of O. Furthermore, by the definition of P x , we have w(x) > w(z) for 
all z € N x . Again, this depends on the fact that the weights of elements in S are the same for all 
^-replacements considered by the algorithm during a given phase. 

The following extension of a theorem from [2] allows us to relate the non-oblivious potentials 
w 2 and w 2 p N s to the weight functions w and W(p xi n x ) for each of our /^-replacements (P X ,N X ). 

Lemma 4.2. For all x G S, and e 6 P x , 

w fp,,N x )( e ) ~ w2 ( Y e ~x)> w(x) ■ (2w {Px:Nx) (e) - w{Y e )) . 

Proof. Let a = ^w(Y e ), and 6, c be such that w(x) = a + b and wip x ,N x )( e ) = a + c (note that b 
and c are not necessarily positive). Then, since e £ P x , every element z in Y e has weight at most 
w(x) = a + b. Furthermore, w(Y e — x) = w(Y e ) — w(x) = a — b. Thus, 

w 2 (Y e -x)= E w ( z ? < E (a + b)w(z) = (a + b)(a-b) (5) 

zdY £ —x z£Y e —x 
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Using ([5]) and our definition of a, b, and c, we have 

w (P x ,N x )( e ) ~ w2 ( Y e -x)- w(x) ■ (2w (Px>Nx) (e) - w{Y e )) 

> (a + c) 2 - (a + 6) (a - b) - (a + 6) (2a + 2c - 2a) = (6 - c) 2 > . □ 

Using Lemma 14.21 we can prove the following lemma, which uses the local optimality of S to 
obtain a lower bound on the weight w(x) of each element x € S. 

Lemma 4.3. For each x G S, w(x) > ^ {2w^ PxNx ^{e) — w{Ye))- 

eeP x 

Proof. Because S is locally optimal with respect to /c-replacements, including in particular (P X ,N X ), 
we must have: 

w 2 (PM (P x )<w 2 (N x ) (6) 

First, we consider the case w(x) = 0. Recall that all the weights produced by the algorithm are 
non-negative. Because w{x) is the largest weight in N x , we must have w(e) = for all e € N x 
and so w 2 (N x ) = 0. Moreover, ([6]) implies that w 2 p N \{P X ) = as well, and so, in particular, 
w (Px,N x )( e ) = 0- The claim then follows. 

Now, suppose that w(x) ^ 0. From (|6|), together with the fact that x £ Y e for all e € P x , and 
the non-negativity of all the weights w, we have: 

w (p*,n*)( P *) < ™ 2 {N X ) < w 2 (x) + £ ™ 2 (Ye - x) • (7) 

edP x 

Rearranging (J7J) using w 2 Px ^ N ^(P x ) = J2 e eP x w fp x ,N x )( e ) we obtain: 

E w U,N x )(z) ~ ™ 2 (Y e -x)< w 2 (x) . (8) 

eeP x 

Applying Lemma 14.21 to each term on the left of (|8|) we have: 

w(x) • (2w (PM (e) - w(Y e )) < ]T wf Px)Nx) (e) - w 2 (Y e - x) < w 2 (x) = w(x) 2 . (9) 

e£P x eeP x 

Dividing by w(x) (recall that w(x) ^ 0) then yields 

{2w {PM (e) - w(Y e )) < w(x) . □ 

e£P x 

We now prove our main result, which gives an upper bound on the locality gap of Algorithm [2j 
Theorem 4.4. (*±2 + e ) f(S) > f(O) 

Proof. Lemma [4.31 gives us one inequality for each i£S. We now add all IS"! inequalities to obtain 

E E ( 2w (PM(e)-w(Y e )) < Y;w(x) . (10) 

xeSeePx xes 
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We have E x( ,s w ( x ) < f( S ) h V ©• Additionally, from @, f{SUP x )-f(S)-\P x \a < J2 eePx w (PM (e) 
for every P x . Thus, (I10p implies 

2j2(f(SUP x ) ~ f(S) - \P x \a) - J2J2 w ( Y e) < f(S) • (11) 

Since {P x }x.es is a partition of 0, (fTTj) is equivalent to 

2^(/(SUP*)-/(S))-2|0|a-^^y e )</(£) . (12) 

We have w(x) > for all i£S, and there are at most k distinct e for which x £ Y e , by property 
[(K2)] of y. Thus, we have 

<fcX>(s)<A:/GS) , 

by (H]). Combining this with (|12p . we obtain 

2 £ (/(S U P x ) - f(S)) - 2\0\a - kf(S) < f(S) (13) 

xGS 

Using again the fact that P is a partition of O, we can apply Lemma |4. II to the remaining sum on 
the left of CEl yielding 

2 (f(S U O) - f(S)) - 2\0\a - kf(S) < f(S) 

which simplifies to 

f(SuO)-\0\a<^f(S) . (14) 
From the definition of a and the optimality of O, we have 

\0\a <na = 5f(S imt ) < 5f(0) . 
Finally, since / is monotone, we have f(SL) O) > f(0). Thus, (|14|) implies: 

(l-8)f(0)<^f(S) , 

which, after expanding the definition of 5 and simplifying, is equivalent to f(0) < (^r + e ) f{S)- 

□ 

Next, we consider the runtime of Algorithm [2l Each iteration requires time O(n) to compute 
the weights for S, plus time to evaluate all potential /c-replacements. There are 0(n, fe + fc ( fc_1 )+ 1 ) = 
0(n k +1 ) such /c-replacements (A,B), and each one can be evaluated in time 0(k 2 ), including 
the computation of the weights W(a,_b)- Thus, the total runtime of Algorithm [2] is 0(lk 2 n k2+1 ), 
where I is the number of improvements it makes. The main difficulty remaining in our analysis is 
showing that Algorithm [2] constantly improves some global quantity, and so I is bounded. Here, 
we show that although the weights w assigned to elements of S change at each iteration, the non- 
oblivious potential w 2 (S), is monotonically increasing. While the preceding analysis of the locality 
gap is valid regardless of the particular ordering -< used to generate the weights, our analysis of 
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the convergence of Algorithm [2] requires that -< be updated at each phase to maintain the relative 
ordering of all elements in the current solution. 

Finally, we consider what happens to the total squared weight w 2 (S) of the current solution 
after applying an ^-replacement (A, B). In order to show that our algorithm terminates, we would 
like to show that this value is strictly increasing. To show this, it is sufficient to show that each 
weight w(x) for x G (S \ B) U A is strictly greater after applying the ^-replacement than the 
corresponding weight before. Unfortunately, the weight assigned to an element is highly sensitive 
to the ordering -< in which elements are considered. Let w be the weight function for solution S, 
and w' be the weight function for solution (S \ B) U A. If x -< y for some x G A, y G S \B, then in 
the updated weight function, we could have w'(y) < w(y), since w' considers the marginal gain of 
y with respect to a set containing x, while w does not (since x G" S). We avoid this phenomenon 
by updating the ordering -< each time an improvement is made. In particular, we ensure that all 
the elements of S and A are considered in the same relative order, but that all of A comes after all 
of S. As we shall show, this ensures that the weights assigned each individual element in the S\B 
and A solution do not decrease after applying the ^-replacement (A, B). 

Lemma 4.5. Suppose that for some k -replacement (A, B) and a > 0, we have w 2 A B \(A) > w 2 {B) + 
a, and Algorithm applies the replacement (A, B) to S to obtain solution T = (S \ B) U A. Let 
ws be the weight function for solution S and wt be the weight function for solution T. Then, 
w 2 (T) > w 2 s (S)+a. 

Proof. After applying the ^-replacement (A, B) to S, we obtain a new current solution T = (S \ 
B) U A and a new ordering -<j>- We now show that for any element x G S \ B, we must have 
w(x) = ws{x) < wt{x) and for any element y G A, we must have wr^m(y) — wt(v)- 

In the first case, let S x be the set of all elements in S that come before x in the ordering -<, and 
similarly let T x be the set of all elements in T that come before x in -<t- Suppose that for some 
element z G T we have z -<t x. Then, since x G S \ B, we must have z -< x. Thus, T x C S x . It 
follows directly from the submodularity of / that 



w(x) = ws(x) 



f(S x + x)- f(S x ) 



o 



a < 



f(T x + x)- f(T x ) 



o 



a 



wt(x) 



In the second case, let A y be the set of all elements of A that come before y in the ordering -<, 
and let T y be the set of all elements of T that come before y in the ordering -<x- Suppose that for 
some element z G T we have z -<t y- Then, since y G A, we must have either z£S\B or z£A 
and z <y. Thus, T y C (S \ B) U A y , and so 



(A,B) 



(y)= 



f((S\B)uA y + y)-f((S\B)uA y ) 
a 



a < 



f{ Ty + y)-f(T y ) 



a = w T (y) 



From the above bounds on w and w^b)^ together with the assumption of the lemma, we now 
have 



w s( s ) = ^2 w s( x ) 2 + ^2 w s(x) 2 < Yl w s( x ) 2 + ^2w(A,B)(y) 2 + « 

xeS\B xeB xeS\B yeA 

< ^ w t{x) 2 + Y2w T {y) 2 + a = Wt(T) + a . □ 



x£S\B 



y£A 
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Theorem 4.6. For any value e € (0,1), Algorithm^ makes at most 0(n 3 e 2 ) improvements. 

Proof. Note submodularity implies that for any element e and any set T C (J, we must have 
/(T + e) — /(T) < /({e}) < /(5Viit)- In particular, for any solution 5 C (/ with associated weight 
function to, we have 

W 2 (S) = J>(e) 2 < |5|/(5 8mi ) 2 < n/(5 m8t ) 2 . 

Consider a given improvement (A,B) applied by the algorithm. Because every weight used in 
the algorithm is a multiple of a, we have w 2 A B \(A) > w 2 (B) only if w 2 A B ^(A) > w (B) + a 2 . Let 
T = (S \ B) U A be the solution resulting from the improvement, and, as in the proof of Lemma 
14.51 let ws be the weight function associated with S and wt be the weight function associated with 
T. For any e > 0, we have a > 0, and hence a 2 > 0. Thus, from Lemma 14.51 after applying the 
improvement must we must have w^iT) > Wg(S) + a 2 . 

Thus, the number of improvements we can make is at most 

nf(S mit ) 2 -^ f{S mit ) 2 ={n _ l) ffiS^tA 2 = {n _ yrf = Q(n 3 6 - 2) n 

Corollary 4.7. For any e > 0, Algorithm^ is a ^r^ + e approximation algorithm, running in time 
0{e~ 2 k 2 n k2+i ). 

5 Open Questions 

We do not currently have an example for which the locality gap of Algorithm [2] can be as bad 
as stated, even for specific /c-exchange systems such as fc-set packing. In the particular case of 
weighted independent set in {k + l)-claw free graphs Berman [2] gives a tight example that shows 
his algorithm can return a set S with ^^w(S) = w(0). His example uses only unit weights, and 
so the non-oblivious potential function is identical to the oblivious one. However, the algorithm 
of Feldman et al. (given here as Algorithm [1]) considers a larger class of improvements than those 
considered by Berman, and so Berman's tight example no longer applies, even in the linear case. 
For the unweighted variant, Hurkens and Schrijver give a lower bound of k/2 + e, where e depends 
on the size of the improvements considered. Because the non-oblivious local search routine performs 
the same as oblivious local search on instances with unit weights (since 1 = l 2 ), this lower bound 
applies to Algorithm Q] in the linear case. From a hardness perspective, the best known bound is the 
Q(k/\n k) NP-hardness result of Hazan, Safra, and Schwartz |14j . for the special case of unweighted 
k-set packing. 

In addition to providing a tight example for our analysis, it would be interesting to see if similar 
techniques could be adapted to apply to more general problems such as matroid k parity in arbitrary 
matroids (here, even an improvement over k for the general linear case would be interesting) or to 
non-monotone submodular functions. A major difficulty with the latter generalization is our proof's 
dependence on the weights' non- negativity, as this assumption no longer holds if our approach is 
applied directly to non-monotone submodular functions. 

Acknowledgment The author thanks Allan Borodin for providing comments on a prelimi- 
nary version of this paper. 
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